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ABSTRACT 


Prognostic and Health Management (PHM) systems often experience delayed fielding 
and lengthened maturation cycles due to their relative immaturity and the fact that they 
are regarded as non-flight critical systems. The national fiscal crisis and rising debt of the 
U.S. have each placed increased scrutiny on military systems acquisition and 
procurement practices. The Defense Department is pushing for greater emphasis on 
fundamental systems engineering practices earlier in the acquisition phase, with the 
expectation of fewer schedule slips and budget overruns. The acquisition of PHM 
systems could also benefit from increased systems engineering rigor early in their 
development. A 2007 directive from the DoD states that PHM systems be implemented 
into current weapon systems equipment, and materiel sustainment programs where 
technically feasible and beneficial. This research examines the definition of PHM 
requirements and a method for developing a solution neutral architecture for PHM 
systems. The thesis also identifies software development practices and acquisition 
processes for military propulsion PHM systems. The conclusion of this research is that 
the Defense Department can deliver the warfighter a capable PHM system on-time and 
within budget through the establishment of better procurement and systems engineering 


practices. 
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EXECUTIVE SUMMARY 


PHM systems are meant to replace legacy schedule-based maintenance systems and it is 
important for acquisition professionals to manage customer expectations. The research 
will show that it is the responsibility of program managers and systems engineers to field 
capable CBM+ systems early in the weapon system life cycle, such that the system and 
the overarching CONOPS can be optimized and matured. Some keys to delivering 


capable PHM systems closer to budget and cost targets are: 


1 Incentivize Program Management to focus on Life Cycle Cost Savings 

2 Involve the end-user beyond the requirements development phase 

3: Manage warfighter expectations to minimize Taguchi losses 

4 Plan to support PHM software throughout the weapon system life cycle 

This research provides a useful reference to program managers and systems 
engineers tasked with the implementation of Prognostic and Health Management (PHM) 
systems. It presents the acquisition of these systems within the framework of systems 
engineering process and also provides acquisition strategies for PHM hardware and 


software products. 


The establishment of stakeholder requirements for PHM systems is enabled by an 
understanding of legacy maintenance practices and an assessment of the technologies 
currently available to replace schedule-based maintenance practices. This research 
examines these maintenance practices and presents a strategy for evaluating PHM 


technologies. 


The next step in this systems engineering analysis of a PHM system is the 
establishment of a notional architecture. This solution neutral approach helps us model 
the system without any pre-conceived notions of how to solve the problem. Once this 
high-level architecture is presented the research discusses software development. Systems 
engineering processes and an evolutionary approach to software development are 


identified as necessary elements to an on-time and on-budget delivery. 
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I. INTRODUCTION 


A. BACKGROUND 


Prior to the F-35 Joint Strike Fighter (JSF) Program, the A-7 Corsair II was the 
last single engine fighter in the Navy tactical aviation fleet. According to 
GlobalSecurity.org, “the A-7 began combat operations in December, 1967, and proved to 
be one of the most effective Navy close support and strike aircraft during the Vietnam 
War [1].” The A-7 was also the platform that marked the beginning of on-board engine 
health monitoring for the Navy. A brief given by Andrew Hess at an NDIA Conference in 
2001 states, “the engine monitoring system (EMS), developed by Navy engineers, 
tracked engine parameters and used two vibration transducers to detect anomalies in the 
mechanical health of the engine. The EMS was a successful technological advancement 
on the A-7 and eventually reduced engine related accident rates by 90% and reduced 


maintenance man hours [2].” 


The Health and Usage Monitoring System (HUMS) developed for military 
helicopters greatly advanced the state of mechanical diagnostics. This system architecture 
was mainly composed of vibration sensors mounted on the propulsion system and 
throughout rotary wing drivetrains. The current engine monitoring system being 
developed for the JSF is known as Prognostics and Health Management (PHM). 
According to their website, “the JSF Program has leveraged advances in sensing 
technologies and processing capabilities to develop an integrated engine monitoring 
system with the ability to detect impending failures well in advance, thus increasing 


fleetwide safety and decreasing operating costs [3].” 


The Navy has had an increased focus and commitment to advancing the 
technological capability of engine monitoring systems in each air system procured since 
the A-7. Figure 1 loosely illustrates the evolution of diagnostic systems in Navy aircraft 


since the late 1960s. 


TEAM Relative Evolution of Mechanical Diagnostic 
=” Capabilities for Navy Aircraft 





Capabilities + Performance 


1990 


Figure 1. | Evolution of EMS in Naval Aviation. From [2] 


A DoD Instruction titled Condition Based Maintenance Plus for Materiel 
Maintenance states that “CBM+ is the application and integration of appropriate 
processes, technologies, and knowledge-based capabilities to improve the reliability and 
maintenance effectiveness of DoD systems and components [4].” The instruction also 
mandates that all acquisition programs must implement a CBM-++ strategy and concept of 
operation (CONOPS) “into current weapon systems, equipment, and materiel sustainment 
programs where technically feasible and beneficial [4].” For the Navy and their aviation 
propulsion systems, the implementation of this DoD Instruction will eliminate scheduled 
inspections of turbomachinery and enable the health assessment of components based on 
their actual usage and life remaining. This, in turn, will lower sustainment costs and 


increase fleet readiness. 


B. PURPOSE 


The largest life cycle cost associated with military aircraft is not the unit price or 
the cost of the multi-year development programs: rather, it is operation and sustainment 
(O&S) costs. The life cycle of military aircraft varies by model, but it is usually no less 
than 25 years. The cost associated with operating and supporting these aircraft is usually 


between 65% to 80% of the entire life cycle cost. 


The DoD has recognized the need to control O&S costs and is taking steps to 
reduce them for future aviation programs. Figure 2 illustrates that O&S comprises the 
largest portion of the weapon system lifecycle. It is, therefore, the area with the most 


potential for cost savings. 
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Figure 2. | Weapon System Life Cycle Illustrates that O&S Represents a Majority of 
Life Cycle Costs. From [5] 





The DoD mandate to implement PHM technologies and a CBM+ strategy are 
meant to specifically target cost savings in this area. The benefits of the system go 


beyond O&S cost savings, however. According to Andrew Hess, “PHM can also enhance 


mission reliability and aircraft safety, eliminate scheduled inspections, eliminate “cannot 
duplicates and retest OK’s,” reduce aircraft downtime, provide opportunistic 


maintenance, and detect incipient faults [6].” 


Over the past decade, there has been a substantial increase in the amount of 
military spending. Figure 4 illustrates the trend in defense spending since 2001. The large 
number of new defense acquisition programs has brought with it a greater scrutiny of the 


cost overruns and schedule delays that have plagued weapon systems acquisition. 
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Figure 3. Defense Spending Profile. From [7] 


The DoD has placed a greater emphasis on fundamental systems engineering 


practices earlier in the acquisition phase, with the expectation of fewer schedule slips and 


4 


budget overruns. Legacy acquisition programs have shown that PHM systems often 
experience delayed fielding and lengthened maturation cycles. The current acquisition 
strategies used by the DoD could benefit from increased systems engineering scrutiny 
and rigor early in their development. The purpose of this research is to investigate how 
PHM systems and architectures have evolved and identify ways to leverage systems 
engineering, program management, and systems acquisition practices to procure and field 


more effective PHM system. 


This thesis is meant to provide Systems Engineers and Program Managers a 
resource for the acquisition of more effective propulsion PHM systems. The research 
outlines processes for establishing requirements, defining a notional architecture and 
developing software for PHM systems and presents them within the systems engineering 
process. It also provides acquisition strategies and lessons learned based on legacy PHM 
systems. These systems are required for many DoD weapons and improved systems 
engineering processes tailored for PHM systems acquisition will improve cost and 


schedule. 


C. RESEARCH QUESTIONS 


This thesis will examine the systems architecture evolution of military aircraft 
propulsion diagnostic systems. The analysis will start with early military aircraft 
diagnostic systems and continue through the current Prognostic and Health Management 
(PHM) systems currently being developed and fielded. The thesis will also identify 
heuristics and strategies to improve acquisition strategies for procuring PHM systems. 
The following research questions will provide a roadmap for answering the overarching 
research question. 

1. How do we establish requirements for propulsion PHM systems? 


2. What process can we use to define a high-level solution neutral PHM 
system architecture? 


Se What systems engineering process can be used to develop software for 
PHM systems? 


4, Can the Defense Department deliver the Warfighter a capable PHM 
system on-time and within budget through the establishment of better 
procurement and systems engineering practices? 


D. BENEFITS OF STUDY 


This research will be used to recommend improvements to the acquisition of 
PHM systems. The research will identify fundamental systems engineering aspects in the 
procurement of PHM systems and offer ways to field more capable systems earlier in the 


weapon system lifecycle. 


E. SCOPE 


The focus of this thesis is limited to propulsion PHM systems for fixed wing 
tactical fighters. This research will employ systems engineering processes and tools to 


identify ways to maximize the value and capability of these PHM systems. 


F. METHODOLOGY 


The needs analysis phase developed by Kossiakoff defines the methodology used 
in approaching the research. Kossiakoff describes the needs analysis as “part of the 
conceptual exploration phase;” it was therefore useful in developing an outline for this 
research [8]. The operational deficiencies and technical opportunities, as illustrated in 


Figure 5, are inputs to the needs analysis. 


Operational System Operational 
Deficiencies Effectiveness 


/ 


Needs Analysis Concept Exploration 


System Studies 





Technology Assessment 
Operational Analysis 


Te A 


Technological System Capabilities 
Opportunities 


Figure 4. Kossiakoff - Needs Analysis. From [8] 


For the purposes of this research, the operational deficiencies represent the legacy 
maintenance practices used to maintain DoD propulsion systems and the technical 
opportunities are the CBM+ systems meant to replace them. The legacy maintenance 
practices are only operationally deficient in the sense that there exists a mandate to 
replace them with technology driven solutions that whereby the DoD can potentially 
realize O&S cost savings. This research will identify methods for leveraging these 
technological opportunities using systems engineering best practices. In Chapter II, a 
requirements analysis of PHM systems is developed. This section uses System Studies to 
examine legacy systems that have implemented PHM systems. The research will also 
perform a Technology Assessment of the PHM sensors and examine how the F-35 
program used this assessment to downselect its sensing suite. The synthesis of these 
concepts provides a foundation to establish some notional requirements for a PHM 
system. In Chapter III, the architecture for this notional system is developed along with 
software development. This system architecture represents the output of the needs 
analysis phase, which then leads to Concept Exploration. The concept in this block is 


represented by the final research question presented in the previous section: Can the 
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Defense Department deliver the Warfighter a capable PHM system on-time and within 
budget through the establishment of better procurement and systems engineering 
practices? Chapter IV will examine these process and practices and identify some of the 
issues associated with the acquisition of PHM systems. Chapter V will compile the 


research and provide heuristics for future acquisition programs. 


This research also follows a framework similar to the PD-21 curriculum at the 
Naval Postgraduate School. The program initially focused on systems engineering 
principles and concepts, while the latter part was dedicated to the author’s chosen 
concentration in systems acquisition. The first three chapters of this research provide a 
systems engineering foundation for PHM systems and the overarching CBM+ 
architecture they are meant to support. The final two chapters integrate these systems 
engineering concepts, but focus on coursework related to the acquisition concentration. 
Overall, the research is meant to explore the design, development, and acquisition of 
PHM systems, while applying the concepts and principles presented during the PD-21 


coursework. 


I. LITERATURE REVIEW 


A. INTRODUCTION 


The research in this thesis builds upon concepts presented in various sources. 
However, the DoD CBM+ guidebook provides the foundation for the requirements and 
architecture sections of this research. The foreword of the guidebook, provided by Jack 
Bell, Deputy Undersecretary of Defense for Logistics and Materiel Readiness, states that 
“it was developed to be an information reference as well as a tool to assist logistics 
managers with CBM+ project development, implementation, and execution [5].” This 
thesis is also meant to be an information reference and tool to assist the developers of 
PHM systems. This research will leverage the processes in the CBM+ Guidebook, 


present them within the systems engineering process and tailor them to PHM Systems. 


According to DoD Instruction 4151.22, “CBM+ is maintenance performed based 
on evidence of need provided by Reliability Centered Maintenance (RCM) analysis and 
other enabling processes and technologies. CBM+ uses a systems engineering approach 
to collect data, enable analysis, and support the decision-making processes for system 
acquisition, sustainment, and operations [4].” The systems engineering approach used by 
CBM+ is also discussed with greater detail in the CBM+ Guidebook. It states that 
“systems engineering is the overarching process that a program team applies to move 
from a required capability to an operationally effective and suitable system. Systems 
engineering processes are applied early in concept refinement, and then continuously 
applied throughout the system’s life cycle. Program managers and life-cycle logisticians 
should consider the effect system development decisions (such as the application of the 
CBM+ strategy) will have on the long-term operational effectiveness and the logistics 
affordability of the system. The cost to implement a system change, including 
supportability enhancements, increases as a program moves further along its life cycle. 
CBM+ has the greatest leverage in the early stages of development, when the program 
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design is most flexible [5].” This research aims to explore the early stages of 


development for PHM systems where flexibility provides system developers the 
9 


opportunity to perform trades on various design criteria. For the purposes of this research, 


these early stages include the requirements and architecture definition. 


The CBM+ guidebook also states that it “presents key elements and 
implementation strategies for achieving incorporation of CBM+ enablers into the DoD 
maintenance process [5].” It is important to understand how PHM fits within the system 
of systems that comprise CBM+. The following sections discuss concepts presented in 
the DoD CBM+ Guidebook as well as several other related sources to provide a baseline 


understanding of the maintenance concepts upon which this research builds. 


B. CONDITION BASED MAINTENANCE PLUS (CBM+) 


The CBM+ guidebook states that “CBM+ is not a single process in itself. It is a 
comprehensive strategy to select, integrate, and focus a number of process improvement 
capabilities, thereby enabling maintenance managers and their customers to attain the 
desired levels of system and equipment readiness in the most cost effective manner 
across the total life cycle of the weapon system. CBM+ includes a variety of interrelated 
and independent capabilities and initiatives—some procedural and some technical that 


can enhance basic maintenance tasks [5].” 


A discussion of components of CBM+, as found in Figure 5, provides necessary 
background for the research presented in subsequent chapters. Reliability centered 
maintenance is another process used by the DoD to enable informed maintenance 
practices. The Office of the Secretary of Defense defines RCM as “a logical, structured 
process used to determine the optimal failure management strategies for any system, 
based on system reliability characteristics and the intended operating context. RCM 
defines what must be done to a system to achieve the desired levels of safety, reliability, 
environmental soundness, and operational readiness, at best cost. RCM is to be applied 


continuously throughout the life cycle of any system [9].” 


In his paper Defining Requirements for Advanced PHM Technologies for Optimal 
Reliability Centered Maintenance, Dr. Richard Millar states that “RCM provides a 


logical decision process for determining optimum maintenance approaches and 
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establishes the evidence of need for both reactive and proactive maintenance. RCM is 
used along with the propulsion system FMECA' to identify capability gaps in the fault 
detection. Given a reasonably representative FMECA and a baseline suite of proven 
PHM tools, a rigorous RCM analysis should yield PHM functional requirements and 
preferred state of the art technical approaches. If this fails to meet weapon system 
objectives, we may need to make tradeoffs between safety, availability, initial and 
recurring costs, and weapon system performance [10].” Figure 5 illustrates the means by 
which CBM and RCM, combined with systems engineering principles and PHM 
capabilities, form CBM+. 


Functionality 


Processes Hardware/ 


Software 
RCM 


CBM 


Design Tools 


Communications 





Figure 5. Various Systems and Processes that Comprise Condition Based 
Maintenance Plus. From [5] 


! FMECA—Failure Mode Effects and Criticality Analysis. Process used to determine the functions, 
functional failures, and failure modes of equipment; and the associated effects, severity, and frequency of 
each failure modes [11]. 
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The CBM+ guidebook states that CBM+ includes, but is not limited to, the following 
examples: 


e Hardware—system health monitoring and management using embedded 
sensors; integrated data bus 


e Software—decision support and analysis capabilities both on and off 
equipment; appropriate use of diagnostics and prognostics; automated 
maintenance information generation and retrieval 


e Design—open system architecture; integration of maintenance and 
logistics information systems; interface with operational systems; 
designing systems that require mini- mum maintenance; enabling 
maintenance decisions based on equipment condition 


° Processes—RCM analysis; a balance of corrective, preventive, and 
predictive maintenance processes; trend-based reliability and process 
improvements; integrated information systems providing logistics system 
response; Serialized Item Management 


° Communications—databases; off-board interactive communication links 


° Tools—integrated electronic technical manuals (i.e., digitized data); 
automatic identification technology; item-unique identification; portable 
maintenance aids; embedded, data-based, interactive training 


e Functionality—low ambiguity fault detection, isolation, and prediction; 
optimized maintenance requirements and reduced logistics support 
footprints; configuration management and asset visibility [5]. 


The Defense Acquisition Guidebook for Systems Engineering states that CBM+ 
is a system of systems (SoS) that presents a fundamental systems engineering problem: 
“the integration of various and disparate technologies and concepts [12].” It defines an 
SoS as a “set or arrangement of systems that results from independent systems integrated 
into a larger system that delivers unique capabilities. Both systems and SoS conform to 
the accepted definition of a system, in that each consists of parts, relationships, and a 


whole that is greater than the sum of its parts [12].” 


C. ESTABLISHING REQUIREMENTS AND ARCHITECTURE FOR PHM 
SYSTEMS 


In 2001, the Defense Acquisition University published a Systems Engineering 
Fundamentals (SEF) book that “provides a basic, conceptual-level description of 


engineering management disciplines that relate to the development and life cycle 
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management of a system [13].” The SEF provides a roadmap to support the development 
of requirements definition for PHM systems. The SEF states that “requirements are the 
primary focus in the systems engineering process because the process’s primary purpose 
is to transform the requirements into designs [13].” Figure 6 represents the systems 
engineering process and illustrates how requirements are the basis of design through 


iterative processes. 


Requirements System Analysis 
Analysis and Control 


(Balance 


Requirements 
Loop 


Functional Analysis 
and Allocation 


Design 
Loop 


Design Synthesis 


Verification 





PROCESS OUTPUT 


Figure 6. _ Illustrates the Systems Engineering Process. From [13] 


To establish stakeholder driven requirements for a PHM system that will enable a 
CBM+ conops, program managers and systems engineers must understand the current 
schedule based maintenance practices. The CBM+ guidebook states that “maintenance 
can be performed using a wide variety of approaches. Two main categories of 


maintenance, reactive and proactive, encompass the full range of options available: 
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e Reactive maintenance (also called corrective maintenance) is performed 
for items that are selected to run to failure or those that fail in an 
unplanned or unscheduled manner. An item may be on a schedule for 
periodic maintenance, but if it fails prematurely, it will require 
maintenance to fix. Reactive maintenance of a reparable item is almost 
always unscheduled in the sense the failure occurred unpredictably. 
Reactive maintenance restores an item to a serviceable condition after the 
failure has occurred. 


e Proactive maintenance is considered either preventive or predictive in 
nature, and the maintenance performed can range from an inspection, test, 
or servicing to an overhaul or complete replacement: 


e Preventive or scheduled maintenance can be based on calendar 
time, equipment operating time, or a cycle (such as number of 
starts, air vehicle landings, rounds fired, or miles driven). 
Preventive maintenance may be either scheduled or unscheduled; 
that is, it is initiated based on predetermined intervals or, 
alternatively, triggered after detection of a condition that may lead 
to failure or degradation of functionality of the weapon, 
equipment, or component. 


e Predictive maintenance can be categorized as either diagnostic or 
prognostic. Diagnostic identifies an impending failure, while 
prognostics add the capability to forecast the remaining equipment 
life. Knowing the remaining life is an obvious benefit to enable 
optimum mission and maintenance planning [5].” 


Figure 7 provides a breakdown of reactive and proactive maintenance approaches. 
Based on this table, it is obvious that the DoD would want to limit reactive maintenance 
on costly man-in-the-loop weapon systems. There are, however, many variables within 
the decision between preventative and predictive maintenance approaches. The predictive 
systems are complex and expensive, but, as discussed above, the DoD has begun to 
pursue this approach due to the opportunity for O&S cost savings. The use of reactive 
and proactive maintenance approaches and how they relate to military propulsion PHM 


systems is presented later in this research. 
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through experience 


. On- and off-system, On- and off-system, 
Ki nd of near-real-time trend real-time trend 
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Figure 7. | Breakdown of Reactive and Proactive Maintenance Approaches. From [5] 





The CBM+ guidebook states that “the most difficult task for the CBM+ 
implementation team may be to correctly match available hardware, software, and 
supporting technology solutions to the requirements of the future maintenance process. 
This task must begin with the documentation of functional requirements [5].” Having 
established a baseline for legacy maintenance practices, the technical assessment of 
available capabilities is the next step in establishing requirements for a PHM system. The 
CBM+ guidebook suggests that a “proof of principle demonstration” is often the best 
approach to evaluating technologies. The guidebook states that “in light of the time and 
funding resources required for CBM+ implementation, it is highly advisable for 
implementers to accomplish small-scale demonstrations of primary CBM+ methods and 
technologies before full-scale implementation. A short-term pilot test that uses equipment 
likely to be used for later full implementation can be a low risk approach to ensuring the 
feasibility and benefits of the desired capabilities. Demonstration of CBM+ planned 
methods and technologies give managers a higher degree of confidence in the likelihood 


of future success [5].” In the following chapter, this research explores a PHM acquisition 
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program that used a proof of principle approach to evaluate PHM technologies and 


discusses how this test enables the informed development of functional requirements. 


Once the requirements are defined for the PHM system, the next step in the 
systems engineering process illustrated in Figure 6 is functional analysis and allocation. 
The SEF states that “system requirements are allocated and defined in sufficient detail to 
provide design and verification criteria to support the integrated system design. This top- 
down process of translating system level requirements into detailed functional and 
performance design criteria include: 


e Defining the system in functional terms, and then decomposing the top- 
level functions into sub-functions. That is, identifying at successively 
lower levels what actions the system has to do, 


e Identifying and defining all internal and external functional interfaces, 


e Identifying functional groupings to minimize and control interfaces 
(functional partitioning), 


e Revisiting the requirements analysis step as necessary to resolve 
functional issues [13].” 


A functional allocation for a propulsion PHM system is performed in Chapter II 


using methods outlined in the SEF to provide a solution neutral architecture. 


The requirements and architecture development in the following chapter will 
build upon these established concepts presented above. The literature research for the 
software development and acquisition strategy is woven into Chapters [TV and V. Much of 
the material in these chapters is based on the PD-21 course work and the Author’s lessons 


learned having worked PHM acquisition at NAVAIR since 2003. 
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Hl. PHM REQUIREMENTS AND ARCHITECTURE 


A. INTRODUCTION 


This chapter explores the definition of requirements for a propulsion PHM system 
within a systems engineering process. In order to frame a discussion about systems 
engineering, it would be useful to start with a commonly accepted definition from 
Professor Gary Langford at the Naval Postgraduate School: 

Systems engineering is a systematic, disciplined approach to define 


stakeholder-driven requirements and to satisfy those with the development 
of functionally driven, synergistic, innovative alternatives [14]. 


This definition can be decomposed to two basic parts: requirements and 
functional allocation. In order to define stakeholder driven requirements for a PHM 
system, it is necessary to examine the legacy maintenance CONOPS. This activity helps 
bridge the gap between legacy maintenance practices and those needed to enable a 
CBM+ conops. This chapter examines schedule-based inspections and maintenance 
actions currently used on legacy propulsion systems. An understanding of these processes 


and procedures provides a foundation for establishing requirements for a PHM system. 


Once the legacy practices are established, this chapter addresses the assessment of 
PHM technologies. One of the most important and fundamental roles of a Systems 
Engineer is to understand both currently available and emerging technologies. Situational 
awareness of emerging technologies will enable a program to incrementally integrate 
capabilities throughout the development of the weapon system. The integration of the 
legacy CONOPS and emerging technologies provides a basis for the discussion of 


requirements and functional allocation for a PHM system. 


This chapter also outlines a high-level approach to establishing systems 
architecture for a PHM system. This solution-neutral approach will apply principles of 


systems architecting to the design of propulsion PHM system. 
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B. LEGACY MAINTENANCE CONOPS 


As stated in Chapter II, the legacy CONOPS for maintaining tactical fighter 
aircraft propulsion systems is either reactive or proactive. The redundant and safety 
critical design of propulsion system components addresses many of the failures that 
would drive reactive maintenance. However, some system faults that require reactive 
maintenance cannot be designed out of a propulsion system, such as foreign object 
damage (FOD)? and maintenance induced faults.3 Despite the occurrence of some 
reactive maintenance, the vast majority of maintenance performed on aircraft propulsion 
systems is proactive. These preventive and predictive inspections generally take place at 
intervals determined by engine flight hours or sorties. Some typical proactive 
maintenance actions for tactical aviation propulsion systems include: borescope 
inspections, duct dives, and oil sampling activities. These activities are labor intensive 
and over the life cycle of an aircraft, these inspections can drive a great deal of O&S 


costs. 


Borescope inspections in legacy aircraft are generally performed based on 
accumulated flight hours. The inspection can be somewhat invasive and time consuming 
on some legacy platforms due to the location of the borescope ports and access panels. 
The inspection uses an optical probe to provide a visual inspection of turbomachinery. It 
is meant to provide maintainers a means to detect any damage that may have occurred to 
engine components in the gas path, A CBM+ CONOPS might eliminate the need to 
perform this inspection based on time intervals and would rely on sensors and parametric 
data trending to drive this maintenance activity based on the actual condition of the 
engine. This will enable the maintainers to make informed decisions as to when they 


should perform borescope inspections. 





2 FOD can be generated from personnel, airport infrastructure (pavements, lights, and signs), the 
environment (wildlife, snow, ice) and the equipment operating on the airfield (aircraft, airport operations 
vehicles, maintenance equipment, fueling trucks, other aircraft servicing equipment, and construction 
equipment) [15]. 


3 Maintenance is essential to aviation safety, yet improper maintenance contributes to a significant 
proportion of aviation accidents and incidents. This is because a small percentage of maintenance tasks are 
performed incorrectly or are omitted due to human error. Examples include parts installed incorrectly, 
missing parts, and the omission of necessary checks [16]. 
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The duct dive is another proactive maintenance inspection used in legacy tactical 
fighters. During this procedure, the maintenance technician climbs through the aircraft 
inlet to inspect the inlet guide vanes and the first few stages of the engine fan, as seen in 
Figure 6. The purpose of the inspection is to visually and tactilely inspect fan blades for 
cracks that may be propagating in the airfoil. This can also be a time-consuming 
maintenance check and there is potential for maintenance-induced damage (tools left 
behind, inlet surface damage, etc.). Similar to the borescope inspection, this time-based 
maintenance would be eliminated in a CBM+ CONOPS and be driven by some 


automated means of impending failure detection. 





Figure 8. —_ Inlet Inspection of E/A 6B and F-16. From [17] 


Propulsion systems on legacy platforms will generally service the lube system 
after each flight. This procedure involves topping off the oil tank and checking magnetic 
chip collectors. The chip collectors provide an indication of machine wear, which can be 
a precursor to component failure (gears, bearings, etc.). Oil samples are also taken at 
regular intervals and analysis of the lubricant is performed to detect wear material and to 
determine the condition of the oil. Ina CBM CONOPS, these tasks would be performed 
according to need rather than scheduled intervals. The lube level would be monitored and 
replenished based on limits set by the system designer, while the chip detection and oil 


analysis would be replaced by an online oil system monitor. 
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In addition to proactive inspections, critical propulsion system components will 
also undergo proactive replacement. Once life-limited parts have reached a particular 
time on wing, they are replaced regardless of their current state of health. This system 
management policy often leads to discarding perfectly good hardware. A condition-based 
maintenance CONOPS, which enables the health assessment of components, based on 
their actual usage and life remaining, would drive down sustainment costs and increase 
fleet readiness. In order to implement a CBM scheme, a PHM system would need to 
sense, detect, and track the health of the propulsion system. PHM would then drive these 
proactive maintenance actions based on the actual condition of the hardware, as opposed 
to the length of operation. An automated means of monitoring hardware would need to be 


implemented in order to drive a condition-based maintenance scheme. 


C; PHM TECHNOLOGY ASSESSMENT 


As stated above, matching available hardware, software, and technologies is one 
of the most difficult tasks for systems engineers. The CBM+ Guidebook also states that 
no combination of technology is likely to provide the “perfect” solution and the team will 
need to make numerous compromises, trading off required capabilities against cost, time, 
and implementation difficulty [5].” The proof of principle demonstration discussed above 
is a useful tool for assessing available technologies for PHM systems and aiding the team 


in arriving at the right combination of technologies. 


The CBM+ guidebook states that “due to time and funding resources required for 
CBM+ implementation, it is highly advisable for implementers to accomplish small-scale 
demonstrations of primary CBM+ methods and technologies before full-scale 
implementation. A short-term pilot test that uses equipment likely to be used for later full 
implementation can be a low risk approach to ensuring the feasibility and benefits of the 


desired capabilities [5].” 


To assess the available and emerging PHM technologies for the F-35 propulsion 
system, the JSF Program performed a sensor evaluation during the Concept Demonstrator 


Phase (CDP). During this test, candidate PHM technologies were placed on an F-100 
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engine during seeded fault testing.4 Figure 7 illustrates some of the candidate 
technologies used during SFET. These technologies were evaluated for their ability to 


detect various failure modes and seeded fault events during the propulsion system test. 
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Figure 9. | PHM Sensors Evaluated During SFET. From [6] 


The results of the test were used to give the systems engineers a baseline 
understanding of the technologies that were available and the Technical Readiness Level 
(TRL) of each sensor. Using the data from the SFET, the program was able to establish 
requirements for the PHM system, requirements that would eventually enable a CBM+ 


CONOPS. The development and maturation of a PHM system, which enables a CBM+ 





3 Seeded fault testing involves an engine test where failure modes are intentionally excited for the 
purpose of evaluating detection or accommodation capabilities of the PHM system or control system [6]. 
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CONOPS, is potentially the longest lead procurement item for modern weapon systems. 
The SFET testing made Program Management aware of the technologies that could be 
integrated in the near term, but they still needed to plan and budget for advances in the 


sensing and computing capabilities that evolve during maturation. 


D. STAKEHOLDER DRIVEN REQUIREMENTS 


Thus far this chapter has established the legacy maintenance procedures for 
military propulsion systems and an approach to executing a technology assessment. 
Systems engineers developing a PHM system can use this information to proceed with 


the development of stakeholder driven requirements. 


There are several different types of requirements identified in the SEF, but since 
the scope of this research is limited to a notional PHM system, the functional 
requirements are the most applicable. The SEF defines functional requirements as “the 
necessary task, action or activity that must be accomplished. It also states that functional 
requirements must be achievable and must reflect a need or objective for which a solution 
is technically achievable [13].” To simplify this definition, we need to establish a 
necessary task with an achievable solution. Within the scope of this research, functional 
requirements are deemed necessary tasks by examining the legacy maintenance 
procedures the PHM system is being designed to replace. This research also examines a 


technical assessment of PHM sensors used to identify an achievable solution. 


Based on the legacy schedule-based maintenance practices identified above, the 
necessary tasks of a notional PHM system are listed across the top of Table 1. These 
tasks are to eliminate scheduled inspections and to provide diagnostic and prognostic 
assessments of the health of the propulsion system. The achievable solutions are listed 


vertically and were taken from the technology assessment outline above in Figure 9. 


22 


cei atse DM tte cel |Iecraccl (pac need 
FOD/DOD Detection Xx xX 
1* Rotor Cracks 
Blocked Fuel Nozzle 
Combustor Erosion x 
Augmentor Failure 
LPT Blade Rub xX 
Oil Contamination 
Carbon Seal Failure 


Gearbox Failure 


>< ies >< Bes 


Bearing Failure 


Nozzle Failure 


< a X< Bee << Rese << ee =< Bes o< Re 
>< esa O< ese O< Bee O< Bec O< ese Ox 


Bowed Rotor 


Table 1. | HM Functions Mapped to Goals 


The necessary tasks and achievable solutions identified in Table 1 are not 
complete, but offer a good basis for the definition of high-level PHM system 
requirements. The purpose of Table 1 is to provide a mapping of tasks to solutions. This 
connectivity provides systems engineers with insight to the design interfaces and enables 
informed decisions regarding requirements. For example, FOD/DOD Detection and 
Combustor Erosion are each mapped to the goal of Borescope. The identification of this 
design interface along with a criticality analysis from the FMECA can offer Systems 


Engineers increased coverage and confidence for the detection of certain failure modes. 


E. PHM SYSTEM ARCHITECTURE 


As stated in Chapter II, the requirements loop of the System Engineering Process 
is followed by Functional Allocation. This section will attempt to refine high-level PHM 
system requirements to into detailed functional and performance design criteria. The 
principles used for this discussion on PHM architecture are based on course material 
provided by Professor John Osmundson, Naval Postgraduate School [18]. 
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To explore the development of a PHM system architecture, it would again be 
useful to frame the discussion with Professor Gary Langford’s definition: 
The system architecture model is a cohesive statement of the system’s 


physical configuration in terms of modules, the information flow between 
them, and interconnects [14]. 


Professor Osmundson states that “the first step in establishing a system 
architecture for PHM is to create a high-level and solution-neutral functional 
decomposition. A solution-neutral characterization of the system will enable the system 
architect to view the entire problem for what it is and not how to solve it. This approach 
also allows the architect to identify the proper solution, which may not necessarily be the 
first solution [18].” Figure 10 illustrates a generic functional decomposition of a PHM 


system. 


PHM System 


——_—_}-____, 


Data Health 


Acquisition Diagnostics Prognostics Management Testability 





Figure 10. PHM Functional Decomposition 


The data acquisition function will be needed to obtain information from the suite 
of onboard sensors that provides data regarding the health of the engine components. This 
data will be processed using tailored diagnostic and prognostic algorithms to provide an 
assessment of the current state of the propulsion system, as well as a prediction of life 
remaining for all life-limited components. The Health Management module will combine 
all the various data sources and use them within an overarching logistics system to 


provide work orders for inspection, repair and replacement of engine components. 
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Finally, the Testability function will provide the capability for the PHM System to 
perform an autonomous built-in test (BIT), to provide verification of the functionality of 


the system. 


The functional decomposition in Figure 10 offers no particular means to achieve a 
system; it is only meant to frame the problem. The next step is to further decompose the 
solution where it helps to start by modeling the system as a single black box, operating on 


material, energy, and signal flows. The black box decomposition is shown in Figure 11. 
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Figure 11. PHM Black Box Decomposition 


To further refine the black box model, it is decomposed two more layers. 
According to Professor Osmundson, this decomposition will “offer insight to the 
interfaces and functionality” of the system [18]. The illustration in Figure 12 outlines the 
refined functional decomposition. The colored lines in Figures 11 and 12 represent the 
transfer of material (blue), energy (red), and data (dashed blue). The material input is 
represented by Pilot Debrief; after the flight the operator feedback is recorded in the 
PHM System. The energy input is represented by the suite of PHM hardware. This 
hardware consists of sensors and electronics used to detect faults in the system, while the 


data path processes this energy to provide input back to the energy path. This 
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interdependency is represented by a feedback loop between the energy and data inputs. 
These three inputs flow to the electronics function and result in the system output: 


System Health Status. 


Input 
Documented 
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Figure 12. Refined Functional Composition 


F. SUMMARY 


The intent of CBM+ systems is to replace legacy maintenance practices. Chapter 
III identified different types of maintenance performed on legacy propulsion systems. 
The most common routine inspections and proactive maintenance actions for these 


systems were borescope inspections, duct dives, and oil sampling. 


The Seeded Fault Engine Test used by the F-35 Program was presented as a 
strategy to assess available technologies when establishing requirements for a PHM 
system. The test designers used PHM technologies to target schedule-based inspections. 
They also used FMECA information to target specific failure modes. This test gave the 
Program Managers a baseline understanding of the PHM technologies that were available 


and the relative reliability and maturity of each sensing system. 
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As discussed in Chapter II, the CBM+ requirements are based on FMECA 
analysis and PHM technologies. An understanding of the legacy maintenance practices, 
along with knowledge of available technologies, will enable acquisition professionals 
within the DoD to define achievable stakeholder driven requirements. Defining 
achievable requirements should be the focus of every system designer. Weapon systems 
that are developed to meet performance levels beyond the current achievable capability 
often suffer schedule and cost impacts. This concept will be further discussed in 


subsequent chapters. 


The CBM+ Guidebook provides methods for establishing requirements and 
Professor Osmundson offers a technique for developing solution neutral architectures. 
These methods were examined and tailored to PHM systems in order provide system 
developers with a reference and tool for guidance during this critical development stage. 
The CBM+ Guidebook also offers several questions that CBM+ program managers and 
systems engineers can use as a checklist for developing CBM+ systems. 


1. Do I have sufficient background information on CBM+ to assess the 
current maintenance program in my organization regarding this strategy? 


Ze Does the CBM+ implementation team fully understand the reasons for 
transition from current maintenance approaches to a CBM-+ environment? 


> Is additional research needed to familiarize myself and team members 
with CBM+ background, policies, technologies, or other relevant 
information [5]? 


These questions can be used within the scope of this research to provide a 
roadmap for developing the requirements for a PHM system. Each of these questions are 
addressed in this thesis with an examination of legacy maintenance practices for military 
propulsion systems and in subsequent chapters with a discussion of strategies for gaining 


program wide support for the implementation of a disruptive technology. 


The Propulsion PHM architecture is a highly integrated and complex system 
designed to enable a CBM+ CONOPS. By its nature, PHM is a system of systems that 
requires integration across each Integrated Product Team (IPT) within the propulsion 
system. The development of the PHM system not only falls to the PHM system 


developers, but also to each component owner across the entire weapon system. The 
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successful implementation of CBM+ requires that the systems architect and systems 
engineers work to provide a comprehensive overarching architecture and traceability for 
each requirement throughout every component and IPT. This chapter offered some 
techniques for developing that architecture through a solution neutral approach consisting 
of various levels of requirements decompositions and the allocation of functions to 


system goals. 
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IV. PHM SOFTWARE DEVELOPMENT 


A. INTRODUCTION 


Once the system architecture has been established, the software design will 
represent much of the work remaining to implement a CBM+ capable system. This 
chapter will discuss the principles of software development related to PHM and identify 


some common practices and lessons learned. 


B. SYSTEMS ENGINEERING PROCESS FOR PHM SOFTWARE 
DEVELOPMENT 


Thus far, the research has been focused on hardware requirements and 
architecture development for PHM systems. The software development activities for 
PHM systems, however, can represent the most potential for cost and schedule overruns. 
According to a GAO report on software acquisition, “a review of five DoD programs 
found that outcomes were mixed for software-intensive acquisitions. The F/A-18 C/D, a 
fighter and attack aircraft, and the Tactical Tomahawk missile had fewer additional cost 
and schedule delays. For these programs, developers used an evolutionary approach, 
disciplined processes, and meaningful metrics. In contrast, the following programs, 
which did not follow these management strategies, experienced schedule delays and cost 
growth: F/A-22, an air dominance aircraft; Space-Based Infrared System, a missile- 
detection satellite system; and Comanche, a multi-mission helicopter [19].” This section 
will examine the evolutionary approach and some of the processes and metrics that 
helped the F/A-18 C/D and Tomahawk missile program deliver software products with 


fewer additional cost and schedule delays. 


The design and development of software products can be characterized as an 
iterative process. This concept is universally applicable software products for PHM 
systems. These systems generally take more time to mature, because not all faults will 
present themselves during testing. While FMECA driven diagnostics systems are often 
comprehensive, there will always be unknown — unknowns. For this reason, the software 


systems engineering design process followed for the development of these modules is 
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most often an Evolutionary Systems Engineering Model. This model allows designers to 
progress through the basic software development life cycle multiple times, adding 
functionality and lessons learned with each successive release. A DoD memorandum on 
software development states that “evolutionary acquisition and spiral development area 
methods will allow us to reduce our cycle time and speed the delivery of advanced 
capability to our warfighters. These approaches are designed to develop and field 


demonstrated technologies for both hardware and software in manageable pieces [20].” 


The illustration in Figure 11 depicts an evolutionary systems engineering model. 
The process begins with user requirements and progresses from design through test. At 
the end of each cycle, feedback or lessons learned from Operations are documented and 
the iteration process begins. After each iteration, the evolutionary process is followed 
while adding functionality and correcting issues identified in previous versions of the 


logic. 
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Figure 13. Evolutionary Systems Engineering Model. From [14] 


The GAO met with industry software developers to “identify disciplined 
approaches to software development [19].” Figure 12 highlights the four-gated reviews 
often used during the software development activity. The processes outlined in this chart 
exist within the evolutionary approach outline in the previous section. According to the 
GAO, “within each phase are key activities that must take place and knowledge, or 
information, that must be attained to pass a review and move to the next phase of 


development [19].” 
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Source: GAO's analysis of leading companies’ software development practices. 


Figure 14. Knowledge Based Software Development Process. From [19] 


The system requirements are established and approved at Gate 1. As discussed in 
Chapter II, the systems engineering process can be decomposed to requirements and 
allocation. Therefore, this is a critical review that will establish the scope of the entire 
project. There will likely be some changes to requirements during the development of 
most software development projects, but the Gate 1 review should establish the expected 
level of effort for the entire program. The GAO states that “software developers typically 
devote about 20% to 30% of their software development time to requirements-setting 
activities. Doing so ensures that developers will be able to provide managers with key 
knowledge at the requirements review gate and show that requirements have been 


properly vetted with the acquirer and that they are achievable and well written [19].” 


The software design review occurs at Gate 2. Figure 14 shows that at this stage 


the software design documents and test plans are completed and reviewed. The GAO 
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states that “‘as with typical hardware reviews, Gate 2 is often broken down by preliminary 
and critical design review. A stable design ensures that all requirements are addressed 


and that components and interfaces are defined [19].” 


Once the software has passed the design reviews the coding begins in preparation 
of Gate 3. The GAO states that “at this stage, the design documents are used for logic 
development by software engineers. The success of this stage often relies upon the ability 
of the software engineer to interpret the requirements established during design. 
Ambiguous design requirements will often lead to re-work and result in added cost and 


schedule delays at this stage [19].” 


The final stage of the process is testing. In their report the GAO states that 
“testing is then performed to uncover defects or gaps in the code. Leading software 
companies we visited develop test plans after requirements are stable and take steps to 


ensure that there are one or more tests for each requirement [19].” 


Another software development process used by DoD is the Work Breakdown 
Structure (WBS). The Military Standard 881A states that “the WBS identifies the product 
to be developed and/or produced. It relates the elements of work to be accomplished to 
each other and to the end product. A WBS can be expressed to any level of detail; 
however, the top three levels are the minimum recommended any program or contract 
needs for reporting purposes unless the items identified are high cost or high risk [21].” 
The illustration in Figure 13 represents a generic WBS for a PHM Software IPT. The 


elements are identified for three levels to gain perspective on the scope of the project. 
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Figure 15. Notional PHM WBS 


The MUIRS analysis is another software procurement process used by the DoD. 
Professor Dave Matthews states that “a MUIRS analysis is performed using the WBS to 
identify critical support functions within the software [22].” The MUIRS Key in Figure 
13 identifies the WBS elements according to their supportability functions. The 
Maintainability function will be the responsibility of the Software Requirements element. 
This element will ensure that all software maintenance requirements are established to 
support the product throughout its life cycle. All upgrades to the software will reside 
within the Software Change Request element. This element will need to establish a 
protocol for new development, testing, and implementation of all new software 
requirements. The Weapon System Integration element will manage all software 


interfaces. There are only a few interfaces listed in the example WBS, but, in reality, the 
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Propulsion PHM software will have many interfaces and any changes to the product 
could have a cascading effect across the weapon system. The software reliability function 
of the MUIRS analysis is identified within both the Fault Detection and Fault Isolation 
WBS. At this level, the performance specification language is developed. For example, 
the Fault Detection and Fault Isolation requirements for reliability could be established at 
90% and 75%, respectively. Finally, the Security & Safety element of the WBS provides 
these functions during development and throughout the sustainment of the diagnostic 


software. 


The Maintainability portion of the MUIRS analysis could be considered to be the 
most critical for PHM software development. Software support is by far this biggest life 
cycle cost driver and the most significant component of system risk. The ability to 
support major software intensive systems is a paramount mission requirement. According 
to Professor Brad Naegle “the future capability of major systems in a net-centric warfare 
environment is totally dependent on the ability to cost-effectively maintain them [23].” 
As stated in Chapter I, the DoD has targeted O&S costs through better systems 
engineering practices and acquisition training. The O&S costs for weapon systems 
represent the 65% to 80% of the life cycle costs of weapon systems. Professor Naegle 
also states that “within that percentage of O&S costs, the cost to maintain software is 
typically between 60% and 80% of the software component total life cycle cost (LCC) 
[23].” Software maintenance, therefore, has the potential to provide tremendous LCC 
savings to the DoD if the acquisition practices and development can be improved. 
Software maintenance costs will improve if we are able to develop and initially field 
more mature software products. The responsibility for delivering mature systems, as 
previously stated, remains with the Program Managers and Systems Engineers within the 


DoD. 


The final characteristic of successful software acquisition programs identified by 
the GAO was the proper use of metrics. The GAO interviewed several commercial 
software development firms to understand the metrics commonly used by industry. They 


identified the following seven metrics for managing software development activities: 
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Cost, Schedule, Size, Requirements, Tests, Defects, and Quality. Using metrics for cost 
and schedule is pretty common in the DoD. Large acquisition programs now require 
some sort of cost and schedule control and reporting system. The Earned Value 
Management System (EVMS) is used by many government agencies and the private 
sector. Program Managers need to be very familiar with EVMS and lean on their Budget 


and Finance Managers for insight. 


The size metric is often referred to as Source Lines of Code (SLOC). SLOC is the 
number of lines of text within source code. SLOC overruns have the potential to drive a 
great deal of cost into a program. The processing and throughput requirements for 
electronics (i.e., FADEC) are partly based on SLOC estimates. The redesign of this 
hardware due to requirements creep and SLOC overruns is very costly. This is an area 
where the evolutionary concept differs for hardware and software, because you cannot 


necessarily evolve hardware incrementally the way you can with software. 


Establishing requirements metrics early in the acquisition phase is an essential 
component of meeting cost and schedule targets. This requirement metric is used not only 
to track the addition of new requirements, but also their timing. This is important because 
changes are easier and less costly to implement in the early stages of development. New 
requirements that arrive late in development will drive greater cost into the program. The 


metric can be used to alert Program Management of these potential cost drivers. 


The final metrics identified by GAO were tests, defects and quality. These metrics 
provide management with feedback regarding the software that is in the latter stages of 
development. Issues identified by these metrics could impede the evolutionary 
development cycle and cause delays with subsequent versions of code and additional 


capabilities. 


C. SUMMARY 


This chapter also discussed the principles of software development related to 
PHM and identified some common practices and lessons learned. The GAO identified 


three industry best practices for the development of software systems: an evolutionary 


36 


approach, disciplined processes, and meaningful metrics. The evolutionary approach 
employed basic systems engineering process to establish requirements, design, build, and 
test the software. The disciplined process included use of gated reviews at each stage of 
software development. This research also identified and explored the use of the WBS and 
MUIRS analysis as a necessary process for software development and sustainability. The 
metrics commonly used by industry for software development were: Cost, Schedule, 
Size, Requirements, Tests, Defects, and Quality. The F/A-18 C/D and Tomahawk Missile 
were identified as two DoD acquisition programs that were able to deliver their products 


near cost and schedule targets by using these industry best-practices. 
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Vv. PHM ACQUISITION STRATEGY 


A. INTRODUCTION 


This chapter will identify some strategies for acquisition managers regarding the 
procurement of PHM systems. These highly integrated and complex systems are a 
relatively recent requirement for implementation on DoD propulsion systems. This 
chapter will discuss a legacy program that was able to field and upgraded PHM system 
and identify some of the lessons learned and best practices related to this successful 
acquisition program. Chapter IV will also examine some of the obstacles facing the 
program managers that procure PHM systems with the DoD. Finally, the chapter will 


wrap up with a discussion of the maturation process for a PHM system. 


B. F/A-18 E/F IMPLEMENTATION OF PHM 


The Innovator’s Dilemma written by Clayton Christensen introduces the concept 
of sustaining and disruptive technologies. An MIT website offers a summary of the book 
and states that “sustaining technologies are technologies that improve product 
performance. These are technologies familiar to most large companies; technologies that 
involve improving a product that has an established role in the market. Disruptive 
technologies are innovations that result in worse product performance, at least in the near 
term. Disruptive technologies occur less frequently; when they do, however, they can 
cause the failure of highly successful companies that are only prepared for sustaining 
technologies [24].” Christensen’s theories were developed based on the application of 
new technologies by businesses competing for market share, but using an abstract 
interpretation of these concepts, the principles of sustaining and disruptive technologies 


can be applied to the implementation of CBM-+ systems within the military. 


For the purpose of this interpretation, sustaining technologies are analogous to the 
legacy CONOPS for aviation maintenance, i.e., schedule-based inspections, schedule- 
based removals, etc. The technologies that enable these maintenance practices have 
consistently been developed and improved over time. The Navy F/A-18 E/F Super 
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Hornet acquisition program is a good example of a sustaining technology improvement 


for Propulsion PHM and can serve to put the application of this theory in context. 


a 





Figure 16. Super Hornet approach aboard CVN 72. Photo taken by Author. 


The US Navy and Boeing collaborated on a paper about the upgrade of the F/A- 
18 propulsion monitoring system. The paper states that “under Navy direction, GE and 
Boeing have enhanced the proven performance of the F/A-18 C/D F404 In-flight Engine 
Condition Monitoring System (IECMS) and produced an Advanced IECMS for the F/A- 
18 E/F tailored for the F414 engine. The advanced IECMS system is fully integrated 
between the engine and airframe and effectively uses available avionics computers and 
interfaces, which contributes to low system weight. This advanced system includes many 
improvements, including: 

e Better aircrew displays and additional cautions/advisories, 


e Additional mission computer resources, 
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° Reliable, new FADEC> with outstanding fault detection and isolation 


capabilities, 
e Improved monitoring hardware installation and signal processing, 
e Expanded memory unit data recording, 
° Addition of an engine-mounted master electrical chip detector, and 
e Additional maintenance codes 


Each of these capabilities contributes to reduced pilot workload and reduced 
aircraft/engine maintenance. The bottom line is that aircraft readiness is improved with 
fewer engine runs, less down time required for troubleshooting, and rapid turnaround 


through onboard diagnostics [25].” 


There were substantial improvements made to the IECMS during the development 
of the Super Hornet propulsion system. The improvements were largely based on 
advances in computing resources and sensor capabilities that were developed as 
sustaining technologies. These technologies gave the maintainers better fault detection 
and isolation capability, but did not substantially change the procedures for maintaining 
the propulsion system. In other words, the improvements were not substantial enough to 
bridge the gap between the legacy maintenance CONOPS and CBM+, but were an 


important step in that direction. 


While the F/A-18 E/F IECMS upgrade was an important step toward the 
implementation of CBM+, it did not represent a paradigm shift in the way the Navy 
maintained the propulsion system. The implementation of a CBM+ system could be 
characterized as a disruptive technology. For CBM+ to become a reality, the Navy will 
still need to rely on advances in sustaining technologies, but the implementation and 
reliance upon these technologies will be disruptive. The greater reliance on new sensing 
technologies could result in degraded performance in the near term, as the systems are 
matured. DoD maintainers are accustomed to performing schedule-based inspections and 
removals. And while the next generation of maintainers that will learn to perform 


condition-based maintenance and value the disruptive technology, the more experienced 





5 FADEC—Full Authority Digital Electronic Control. 
4] 


maintainers will likely resist the change, particularly in the period during which the 
system is being matured and product performance decreases, 1.e., false alarms, missed 


detects, etc. 


The MIT review of Christensen’s work states that “disruptive technologies cause 
problems because they do not initially satisfy the demands of even the high end of the 
market. Because of that, large companies choose to overlook disruptive technologies until 


they become more attractive profit-wise [24].” This is illustrated in Figure 17. 
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Figure 17. Disruptive Vs. Sustaining Technologies. From [24] 


The Taguchi definition of quality and his theories regarding losses to society are 
relevant to this discussion of CBM+ as a disruptive technology. Taguchi defined quality 
as “the loss a product causes to society after being shipped, other than losses caused by 
its intrinsic function [26].” This theory is analogous and applicable to the development of 
CBM+ systems within the military. These systems are fundamentally designed to enable 
maintainers, but when not properly implemented or designed, they can be burdensome 
and unreliable. As previously stated, the use of PHM systems to drive all maintenance is 
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a huge paradigm shift within the DoD and change is not always welcome. There is a 
natural reluctance and skepticism within the maintenance community to rely on 
automatic fault detection systems to drive maintenance actions. Unsuccessful attempts in 
the past to implement these systems have led to a perception among some maintainers 
(society) that condition-based maintenance may not be an attainable goal. It is the 
responsibility of Program Managers and Systems Engineers to field capable CBM+ 
systems early in the weapon system life cycle, such that the system and the overarching 


CONOPS can be optimized and matured. 


C. PHM MATURATION PROCESS 


PHM design, development, and maturation is a unique process and different than 
all other components on the propulsion system. The PHM System and the associated 
logistics infrastructure that it supports are, potentially, the longest lead development 
items associated with the acquisition of military propulsion systems. The diagnostic 
sensors and other hardware associated with the PHM system is designed, developed, 
tested, and matured along with all of the propulsion hardware. However, this hardware 
only provides a source of data. The software and algorithms that enable a CBM+ system 
are developed through an iterative process long after the propulsion system has gone into 


production. 


As discussed previously, the technologies that enable a CBM+ CONOPS are 
relatively immature. The propulsion system FMECA provides designers with insight as to 
the failure modes the system should target, but the verification and validation of the 
ability to detect these actual failures could, in some cases, occur long after the system is 
fielded. Ideally, PHM system designers would like to perform seeded fault testing to 
validate their capability to detect faults throughout the propulsion system, i.e., the F100 
SFET testing discussed in Chapter II. However, this testing is cost prohibitive in that it 
often leads to damaging expensive hardware for the sake of maturing a non-flight critical 
system. Therefore, this is an implausible approach for most acquisition programs. The 
normal timeline of verification and validation for fault detection and fault isolation 


software extends beyond the development phase and well into production. Although this 
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involves delivering an immature product to the field, many of the failures that the PHM 
system is designed to detect will present themselves during the course of normal 
operation. It is also critical to update and maintain a current and valid FMECA as 
hardware changes occur. The validity of this data is essential for RCM and CBM+ 


systems. 


The Taguchi loss analogy cited in Chapter II stated that delivering an immature 
PHM product to the Services could have an ill effect on the perception and customer 
satisfaction with PHM. However, due to the unique requirements and long lead 
maturation of the PHM system, the end users actually become part of the design team. 
Once initially deployed, these maintainers will be able to provide the system designers 
with feedback regarding the operation of the system and areas for improvement. It is, 
however, critical that the Program Manager and PHM designers deliver a baseline 
product that is representative of the final product with intermediate functionality. If the 
system is deemed unreliable by the end user, the end-user will revert to legacy 
maintenance procedures and the feedback loop necessary to correct inherent system 


design flaws will be gone. 


D. PROGRAM MANAGEMENT DILEMMA 


Professor Dave Matthews stated that “the number one responsibility of a Program 
Manager is to make a program viable from a cost and schedule perspective [22].” History 
has shown that weapon systems acquisition budgets and schedules often suffer due to the 
unknown unknowns. The DoD procures some of the most technically complex systems 
designed by man, and there are inevitable flaws that will present themselves throughout 
the design, build, and test phase of the acquisition programs. This concept can be applied 
broadly to weapon systems or specifically to the development and maturation of 


Propulsion CBM+ systems. 


The program management dilemma with regard to the procurement of PHM 
systems pertains to incentive. With all of the pressure placed on Program Managers to 


make a program viable, there is often no sense of urgency or incentive to focus on 
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systems that are in place to provide LCC savings many years into the program. Although 
this research has cited numerous reasons for providing PHM capabilities as early in the 
program as possible, it has become the norm for Programs to mortgage non-flight critical 
systems to make their Program viable. The justification for doing this is perfectly 
reasonable. There are political pressures that surround defense acquisition programs and 
the technical challenges that the programs often face drive program realignment activities 
that focus on the near-term must haves. However, deferring PHM capabilities comes at 
the cost of both LCC and increased development costs, due to the fact that added 
capabilities are always more expensive to implement later in a program life cycle. The 
DoD is currently focused on these sustainment cost savings, but these savings will never 
be realized for PHM systems without programmatic incentives for acquisition managers 
to focus on LCC. Without some kind of incentive, the costs that any Program Manager 
will focus on are the here and now costs to solve today’s technical issues. The LCC 
savings realized by the implementation of a CBM+ system will likely not be realized 
until well after the Program Manager has moved on or retired. These LCC savings should 
be tracked and pursued from day one; currently, they are not. Program Managers are 
forced to be near-sighted with regard to cost and schedule; this comes at the detriment of 


CBM+ systems. 


At the program management level, supportability and logistics issues are often 
deferred until later in the acquisition program due to limited resources and the redesign of 
more critical hardware and software systems. The prioritization of CBM+ through 
programmatic incentives will not be a trivial policy to implement for the DoD, but it 
needs to be a priority if the LCC savings associated with these systems are to become a 


reality. 


E. SUMMARY 


PHM systems are often introduced with a performance degradation compared to 
the legacy maintenance practices they are meant to improve. Disruptive technologies 
have the potential to transform the way an organization operates and they offer a great 


deal of upside potential with respect to O&S costs. The acquisition professionals must 
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learn to manage not only the effective procurement of these systems, but also the 
expectations of the end user. These expectations were also explored in a discussion of the 
Taguchi Loss Theory. The acquisition managers must also prepare a mitigation strategy 
for the potential perceived quality losses during the introduction of a disruptive 


technology. 


The maturation and software maintenance of the PHM system is likely to last as 
long as the weapon system on which it resides. The end user becomes an integral part of 
the design loop throughout this lifecycle of this product. Once again, the Taguchi losses 
experienced with this disruptive technology must be managed by developers and Program 


Managers. 


If the Life Cycle Cost savings associated with PHM system are to be realized, the 
DoD must incentivize Program Managers to actively attack the enabling technologies. 
These technologies are often an afterthought due to the need to address more immediate 


technical concerns. 
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VI. CONCLUSION 


A. INTRODUCTION 


The purpose of this research was to provide Systems Engineers and Program 
Managers a resource for the acquisition of propulsion PHM systems. The thesis outlines 
processes for establishing requirements, defining a notional architecture and developing 
software for PHM systems and presents them within the systems engineering process. It 
also provides acquisition strategies and lessons learned based on legacy PHM systems. 
The goal of the research is to enable the system developers to use this reference to 
improve cost and schedule delivery of capable PHM systems throughout the acquisition 


life cycle. 


B. RESEARCH SUMMARY 


The research began with a background examination of military propulsion PHM 
systems and the their evolution from the diagnostic systems first fielded on the A-7 
Corsair to the CBM+ systems being developed for the F-35 JSF. It also identified the 
DoD directive that mandates the implementation of CBM+ strategies. This mandate was 
an important step in the evolution of CBM+, but in order to be successfully implemented, 
there must be a sense of ownership throughout each level of the program. PHM is unique 
in that it touches each and every IPT within the weapon system. Its development, 


therefore, is a program wide responsibility. 


There are many different systems engineering process models, but, generally 
speaking, they each start with the establishment of system requirements. In the DoD, 
requirements for weapon systems are commonly defined by the user community. For 
PHM systems, it is important to focus on requirements related to the schedule-based 
maintenance actions we are seeking to transition to a condition-based CONOPS. The 
logical first step was to examine current schedule-based maintenance actions. This 


activity provides a baseline understanding for the requirements analysis needed to be 
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performed by any program planning to implement CBM+. The use of RCM tools and the 
FMECA also provide an important resource for identifying capability gaps and 


addressing them with available PHM technologies. 


The definition of requirements was followed by a high-level approach to 
establishing an architecture for a generic PHM system. This method provides a solution 
neutral characterization of the problem rather than a way to solve the problem. Once the 
high-level architecture is identified, it is decomposed to the basic elements, which add 


resolution by establishing flows material, energy, and data. 


There is no one-size-fits-all approach to software development and acquisition. 
This is particularly true for the PHM software procurement. It is important to focus on the 
established processes and metrics that have been established and proven by previous 
successful acquisition programs. Research performed by the GAO established some 
lessons learned for software development. The report suggests some methods the DoD 
could employ to leverage industry best-practices for delivering quality software products 
within time and cost constraints. The use of an evolutionary approach, disciplined 
processes, and meaningful metrics were identified as key enablers to successful software 
acquisition programs, such as the F/A-18 C/D and Tomahawk Missile Program. Figure 


18 highlights these management practices and how they relate to program goals. 
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Successful outcomes: 
Meeting cost, 
schedule, 
performance, 
and quality goals 









Useful 
metrics 





Disciplined development 
processes 








Manageable development environment 


Source: GAO's analysis of leading companies’ software development practices. 


Figure 18. Key Management Practices. From [19] 


PHM was presented as a disruptive technology that has the ability to revolutionize 
the way the military maintains weapon systems and provide significant LCC savings. 
However, as with any disruptive technology, there will be a period of learning and a need 
to overcome the perceived or actual quality loss by society, as outlined by Taugchi. The 
period of transition from design and development to initial demonstration is critical for 
PHM technologies to be matured. It is important for Program Managers within the DoD 


to defend PHM systems and to manage expectations within the user community. 


Most large companies are adept at turning sustaining technology challenges into 
achievements, the F/A-18 E/F IECMS upgrade is a good example of this. IECMS was a 
sustaining tech upgrade that improved the F404 engine, which had an established role in 
the market. Each of the acquisition programs discussed in this chapter used a systems 


engineering approach to the establishment of requirements for a propulsion PHM system. 
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The difference in the way that the F/A-18 and F-35 programs approached their respective 
evaluation of candidate technologies is obviously much different. The F/A-18 E/F was an 
established platform that was upgrading their current diagnostic capabilities based on 
needs established by the user, whereas the F-35 program used the SFET program to build 
a CBM+ system from the ground up. The JSF Program had a need to establish an 


understanding of the technologies available at that time. 


In each case, the respective Programs started with a customer need and followed 
the systems engineering process to deliver a product. Figure 19 identifies a basic systems 
engineering process V- model commonly used for the development of weapon systems. 
The discussion of the Super Hornet and F-35 Programs focused on the first two stages of 


the model, the Problem Decomposition phase and the System Design phase, respectively. 






System Integration, 


Problem Decomposition Verification, & Validation 


Component Integration, 


Sys Ded Verification, & Validation 


Component Production 


Figure 19. Systems Engineering Process V-Model. From [14] 


The F/A-18 discussion focused on the Problem Decomposition step in the V- 
model. The team identified maintenance issues in the fleet that could be eliminated or 
improved with the IECMS upgrade. They interviewed the users to identify improvements 
and establish requirements to increase maintainability and reliability of the F414. The F- 
35 example was focused on the Systems Design phase of the V-model. This phase 
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occurred after the Problem Decomposition had been performed and systems engineers 
had identified the maintenance actions they wished to target. To begin the systems design 
phase, they performed the SFET to provide an analysis of alternatives and an assessment 


of the TRL of existing PHM technologies. 


Once the PHM system is fielded, the maturation process begins. The support of 
the PHM software will last as long as the weapon system on which it resides. The 
maturation of this system is unique and will often involve the end user more than other 
systems. Care must be taken to manage the Taguchi losses experienced by maintenance 


personnel with regard to immature software products. 


The DoD is aggressively looking for ways to reduce its budget and weapon 
system life cycle costs are a prime target. CBM+ systems are designed to provide LCC 
savings, but the technology is still relatively immature and often not the focus of Program 
Managers. The DoD needs to incentivize its Program Managers to focus on the 
implementation of CBM+ systems earlier in the weapon system development life cycle in 


order to capitalize on the LCC savings they provide. 


Cc. CONCLUSION AND LESSONS LEARNED 


The development of PHM will not succeed without commitment from all levels of 
the Program. The Program Management must be willing to defend the system during the 
inevitable cost-cutting exercises. The system developers must have a sense of 
responsibility for the design and integration activities that cross all IPT’s. All of the IPT’s 
have a stake in the development of the PHM system. The hardware and software 
configuration management and integration activities are extremely complex and errors 


will have a cascading effect across the weapon system. 


Program Management and system developers are responsible for not only 
designing systems according to the warfighter’s requirements, but also managing their 


expectations, particularly when delivering a disruptive technology with intermediate 


51 


functionality. Programs must also account for the budget necessary to maintain PHM 
systems throughout their lifecycle. The Program could consider the establishment of a 


Software Maintenance IPT at some point during the acquisition phase. 


The program manager and logic designers should be aware of and identify risks 
related to the development of PHM software. The following are some common risks 
associated with the PHM software development activity: 


e There is an inherent risk in developing software for safety critical control 
components. The risk is especially high when working with airborne 
weapon systems. The fault detection and accommodation logic must be 
tested, verified and validated to a high standard. 


e Cost and schedule are always significant risk factors during software 
development. The WBS for these programs is such that one IPT depends 
on a product from another IPT. There is a risk of creeping requirements. 
The evolutionary model is used to mitigate this risk, but a large change in 
project scope cannot be absorbed or accommodated by any systems 
engineering model. 


e Staffing issues and the learning curve associated with developing 
complicated software systems can place a large amount of risk on 
programs. 

e The test benches used for Verification and Validation (V&V) testing will 


often run at 100% capacity during peak times. A shortage of equipment to 
test the logic will create a backlog and eventually schedule delays. 


e Configuration control boards will have their own set of priorities. For 
example changes to the Control logic will get higher priority than changes 
to the Diagnostic logic. Excessive development issues with higher priority 
modules will often delay the development of less critical modules. 


Tight collaboration between the customer and the developer is a critical aspect of 
controlling requirements, cost, schedule, and quality during software projects. A 
customer presence at the development site can provide autonomy and a fast turn-around 
when challenges inevitably present themselves. For example, derived requirements will 
often have some ambiguity and leave room for interpretation. This interpretation could 
impact project cost and schedule. However, having a government representative with 
decision making authority integrated with the development team can provide timely 


guidance and potentially eliminate costly rework and delays. 
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As stated above, the DoD is in the business of making the difficult seem easy. 
Schedule delays and cost overruns associated with developing high-tech weapon systems 
are nearly inevitable. These vast and complex systems are bound to experience shortfalls 
and unexpected failures that drive redesign activities. However, it is not unreasonable to 
expect better performance from the DoD and the contractors that develop the weapon 


systems. 


This research provided examples of acquisition and development programs that 
have used systems engineering processes and industry best practices to achieve better 
results. Along with these documented improvements used on legacy programs, the 
research also identified several other success oriented criteria for delivering capable PHM 


systems closer to budget and cost targets, such as: 


e Incentivize Program Management to focus on Life Cycle Cost Savings 
e Involve the end-user beyond the requirements development phase 

e Manage warfighter expectations to minimize Taguchi losses 

e Plan to support PHM software throughout the weapon system life cycle 
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